Visionary Health: An AI-Based Real-Time Medical Consultation System Using Image and Text Input

Abstract

This paper introduces Visionary Health, a dual-mode AI-powered consultation system that provides real-time, preliminary medical assessments using image and text input. Leveraging deep learning and natural language processing (NLP), the system enables users to obtain diagnostic support by uploading images of symptoms or entering a text description. The image-based module utilizes a convolutional neural network (CNN) trained on dermatological data, while the text-based module employs transformer-based models to understand symptom narratives. This flexible approach improves access to healthcare guidance in remote or underserved areas. Evaluation results suggest strong performance across both modules, indicating the system’s promise as a scalable tool for digital health triage.

Introduction

Overview:
Visionary Health is an AI-based diagnostic platform designed to support early detection of skin and eye conditions, especially in low-resource settings. It offers real-time analysis using two modes: image-based and text-based diagnosis.

Key Features:

• Dual Diagnostic Pipelines:

  • Image-Based: Uses Convolutional Neural Networks (CNNs) trained on dermatological datasets to classify skin diseases.

  • Text-Based: Employs a fine-tuned transformer (BERT variant) to interpret user-described symptoms and generate basic medical advice.

• User Interface: Simple and accessible, suitable for non-technical users.

Methodology:

1. Image-Based Diagnosis:

• Preprocessing: Includes resizing, normalization, and augmentation.

• CNN Architecture: Uses layers like convolution, pooling, dense, and softmax. Transfer learning with models like ResNet50 or MobileNet improves accuracy.

• Output: Predicts condition and provides basic prevention and treatment suggestions.

2. Text-Based Diagnosis:

• Input Processing: Tokenization, stopword removal, and lemmatization.

• NLP Model: Lightweight BERT model trained on medical Q&A data.

• Output: Classifies symptoms, identifies keywords, and generates relevant advice.

3. Integration:

• Web-based platform built using Django, TensorFlow, and HuggingFace.

• Users can input via image or text, with the system handling routing and inference.

Results:

Image-Based Module:

• Accuracy: 96.3% (training), 91.7% (validation)

• F1-Score: 91.6%

• Some misclassifications due to visually similar conditions.

Text-Based Module:

• Intent Accuracy: 89.2%

• Relevance Score: 87%

• Average Response Time: 1.5 seconds

• Minor issues with vague or ungrammatical input.

User Feedback (30 participants):

• Ease of Use: 4.5/5

• Satisfaction: 4.2/5

• 87% would recommend the platform.

Conclusion

This study introduces Visionary Health, an AI-enabled consultation system that operates in two modesanalyzing images and interpreting textto deliver prompt, initial medical assessments. The system demonstrates strong performance in classifying common skin diseases using CNN models and effectively interpreting symptom descriptions using transformer-based NLP techniques. Its user-friendly interface and flexible input modes make it especially suitable for deployment in rural or under-resourced regions where access to specialized healthcare professionals is limited.

References

[1] Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., &Thrun, S. (2017). Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542(7639), 115–118. [2] Prasad, M., &Acharya, U. R. (2017). Automatic diagnosis of skin lesions using deep learning. Computers in Biology and Medicine, 80, 52-61. [3] Rajpurkar, P., et al. (2018). AI for healthcare: Diagnosis using NLP models on clinical notes. arXiv preprint arXiv:1802.05319. [4] Buda, M., Maki, A., &Mazurowski, M. A. (2018). A systematic study of the class imbalance problem in convolutional neural networks. Neural Networks, 106, 249–259. [5] World Health Organization (WHO). (2022). Skin infections: Global health burden and prevention. WHO Report. [6] Jain, A., Gupta, R., & Kumar, S. (2021). AI-driven dermatological diagnosis for common skin conditions: A systematic review. International Journal of Dermatology AI, 12(3), 45-58. [7] Wang, L., Chen, Y., & Zhang, H. (2021). Machine learning approaches for detecting mild dermatological conditions using smartphone images. Computers in Biology and Medicine, 134, 104-112. [8] Patel, M., Singh, R., &Verma, P. (2021). NLP-based symptom analysis for minor skin infections: A comparative study. Journal of Medical AI Research, 9(2), 78-90. [9] Lee, J., Park, S., & Kim, H. (2021). Deep learning models for early detection of acne and eczema using mobile applications. AI in Healthcare, 15(4), 102-115. [10] Rodriguez, C., Torres, M., & Garcia, F. (2021). AI-assisted diagnosis of fungal skin infections: A review of recent advancements. Biomedical AI Journal, 7(1), 33-47.

Copyright

Copyright © 2025 Mr. Ranjeet Singh, Rajesh , Nikhil Rai, Suraj Kumar. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.